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ABSTRACT 



This document reports on the evaluation study of the Federal 
Coordinating Council for Science, Engineering, and Technology (FCCSET) 1993 
Summer Institutes that provided opportunities for teachers from around the 
country to participate in hands-on education programs in areas such as 
environmental and ecological studies, material science, space research, and 
alternative energy and conservation studies. Part I of the report details the 
findings of the evaluation study. The Institutes provided significant 
enrichment in science and related content areas but did not explore 
innovations in instruction or reflect current research on theories of 
learning. Part II of this report addresses the follow-up projects that were 
intended to support teachers as they transferred their new knowledge to the 
classroom and provide them with additional materials and equipment. Part III 
details the recommendations resulting from the evaluation study of the Summer 
Institutes and the follow-up projects. Recommendation themes include program 
administration, promoting a new vision of classroom teaching and learning, 
teacher development and program activities, follow-up, teacher leadership and 
responsibility, and program evaluation. Appendices contain data collection 
methods and survey instruments . ( JRH) 
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Executive Summary 



EXECUTIVE SUMMARY 



This document is a report on the evaluation study 1 of the Federal Coordinating 
Council for Science, Engineering, and Technology (FCCSET) 1993 Summer 
Institutes. Under the general supervision of the U.S. Department of Energy (DOE), 

15 Summer Institutes for teachers were held across the U.S. in the summer of 1993. 
The Summer Teacher Enhancement Programs (STEP), known as Summer Institutes, 
provided opportunities for teachers from around the country to study at federal 
facilities. The participants spent four weeks in hands-on education programs in such 
varied areas as: environmental and ecological studies, material science, space 
research, and alternative energy and conservation. All the Institutes had the goal of 
improving middle school/junior high and/or high school science learning by enhancing 
the science knowledge of teachers through in-depth training. The specific topics 
covered varied from site to site. These Summer Institutes represented a significant 
federal investment in teacher professional development, focusing on increasing the 
knowledge base of 560 teachers. 

The federal facilities’ employees who organized and carried out the Summer Institutes 
included education specialists, scientists, and technical staff. In a number of cases, 
outside presenters drawn from higher education, other federal facilities, or state 
agencies, provided expertise that was not available within the Lab itself. 

Participating teachers were chosen from pools of applicants either within the state in 
which the Lab was located or within a multi-state region. The time line between 
funding and start-up was very short, which meant the pool of applicants was quite 
small at many of the sites, resulting in low enrollments at some sites. The majority 
of participants were high school science teachers, with the next largest group being 
middle school/junior high teachers. Some Institutes included elementary teachers; one 
Institute had a majority of participants from elementary schools. 

Part I of this report details the findings of the evaluation study conducted during the 
Summer of 1993. All 15 Summer Institute sites were visited by staff or associates of 
the National Center for Improving Science Education (NCISE). Observations, data 
collection, and interviews conducted by this team followed a "Template for Teacher 



’This study evaluated Summer Institutes for teachers that were initiated by the Federal 
Coordinating Council for Science, Engineering, and Technology (FCCSET), now the 
National Science and Technology Council (NSTC), and its committee on Education and 
Training. Under the guidance of the Subcommittee for Excellence in Science, Mathematics 
and Engineering Education and sponsored by the National Science Foundation (NSF) the 
study was designed and monitored by the Dissemination and Evaluation Working Group for 
the NSTC and carried out by the National Center for Improving Science Education (NCISE), 
at The NETWORK, Inc. 
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Development Programs" created by NCISE in collaboration with DOE staff. This 
format enabled uniform observations of the Summer Institutes and their participants. 

All of the 15 Summer Institutes carried out follow-up work with some or all of the 
teachers who attended the summer sessions. The follow-up was intended to support 
teachers as they transferred their new knowledge to the classroom and to provide 
them with additional materials and equipment. Part II of the report addresses these 
efforts by the various projects. Follow-up was observed during eight site visits to the 
projects that included in-person interviews with teacher participants and Institute staff 
during the 1993-94 school year. In addition, telephone interviews with 63 teacher 
participants and all 15 Institute Directors were used to gather information concerning 
follow-up. 

Part III of the report details the recommendations resulting from NCISE’ s Evaluation 
Study of both the Summer Institutes which occurred in 1993 and their follow-up 
activities that subsequently took place in 1993 and 1994. 

GENERAL FINDINGS FROM THE EVALUATION STUDY OF THE 1993 
SUMMER INSTITUTES 

• The unique role that the federal Labs and facilities play in cutting-edge 
research in science, mathematics, and engineering technology and the wealth 
of knowledge located at these sites provided participating science and 
mathematics teachers with the opportunity to engage in the latest developments 
in science and contemporary research. 

• The 15 Summer Institutes drew upon the substantial human and physical 
resources at the federal facilities where they were held. Some of the Labs were 
better suited than others to support the participating teachers full engagement 
in the research. This depended upon the level of complexity of the science 
being studied and the accessibility of the research facility itself. If laboratory 
spaces were off limits because of hazards such as radioactivity or operational 
needs such as clean rooms, it became harder for teachers to experience and 
comprehend the work of the facility. 

• The Institutes provided significant enrichment in science and related content 
areas, updating teachers’ knowledge base in such varied areas as: 
environmental and ecological studies, material science, space research, and 
alternative energy and conservation. 

• Institute organizers and staff did not clearly articulate a vision of what 
effective science and mathematics teaching should look like in the participating 
teachers classrooms, after the Institute. Institutes, while well-regarded by 
participants, generally did not explore innovations in instruction or reflect 
current research on theories of learning. The Institutes tended to emphasize 
science content, often in relatively narrow fields. While the exposure to such 
content was either a new or enriching experience for the participating teachers, 
they were not consistently offered a means to translate that to effective 
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classroom practice. In fact, very few Institute instructors had experience in 
schools or were knowledgeable about the current research on learning and 
methods of science instruction. 

• Lead time for recruitment and enrollment in the Institutes was inadequate, 
through no fault of the Institutes themselves. 

• Participants reflected the regional population in terms of ethnicity, urban/rural, 
and income levels, since participants were generally drawn from the 
geographic region of the federal Lab involved. 

FINDINGS FROM THE STUDY OF FOLLOW-UP ACTIVITIES CONDUCTED 
DURING THE SPRING AND SUMMER OF 1994 

• Every Institute planned some type of follow-up, but many felt unable to 
provide extensive follow-up due to limited resources and staff time. Lab and 
facilities staff expressed their determination to make provisions for more 
intensive follow-up in future Summer Institutes. 

• Teachers who attended Institutes that incorporated follow-up as an integral part 
of their complete program tended to be more active in incorporating material 
learned during the summer than those teachers who attended Institutes that did 
not place a strong emphasis on follow-up. During at least four of the follow- 
up site visits, and in 15 % of all the teacher telephone interviews, participants 
spoke of the significance of the follow-up process and the enthusiasm 
generated. 

• Even when follow-up was a minor component of the Institute, it was helpful to 
the teachers who participated. Several individuals pointed to the value of 
informal, teacher-to-teacher networking that was made possible by the 
Institutes and reinforced by the follow-up process. 

• Most of the Institutes were not able to generate a sense of on-going 
connectedness or partnership with their 1993 teacher participants because they 
did not have the ability to create on-going networks of practitioners, either 
through repeated face-to-face meetings or telecommunications. 

• Electronic communication, proposed or assumed by all of the 15 Institutes, 
was limited largely because of technological limitations on the part of 
participants’ schools. 

RECOMMENDATIONS 

Specific recommendations for the Funding Agency and those administering the 
Summer Institutes are grouped under the respective components of the "Template for 
Teacher Development Programs" in Part III of the report. They are summarized 
thematically in this executive summary. 
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Program Administration 

Continue the funding of teacher enhancement Summer Institutes in the Federal 
Laboratories and facilities. However, explore better targeting of participants for the 
Institutes, both in background and roles to be played in their schools matched to 
Institute goals. Make certain that there is adequate time for recruiting participants 
and planning of the Institutes. 

Promoting a New Vision of Classroom Teaching and Learning 

Articulate and communicate a vision of how learning and teaching of science can look 
in the classroom that reflects emerging national standards. Involve teachers and 
external presenters, as well as staff, in planning and communicating this vision. 

Teacher Development Program Activities 

Model effective teaching practices in the Institutes that participants will be expected to 
use with their students. Provide a range of experiences for the participants that will 
encourage creativity and a spirit of cooperative inquiry. Formats other than 20 day 
Institutes should be considered to foster these goals. 

Uniqueness of the DOE Laboratories and Other Federal Facilities 

Continue to share with the science and mathematics teachers the unique science 
research skills possessed by the staff in the Labs and other federal facilities. 

Follow-up 

Plan effective continuous follow-up that ensures a lasting relationship between the 
participants and the Lab and other federal facilities staff. Follow-up should include 
hands-on activities easily transferred to the classroom. Be certain that mechanisms 
for follow-up, including telecommunications are available for use by participants in 
their schools. 

Teacher Leadership and Responsibility 

Actively engage the participating teachers in the Institute’s planning and follow-up 
process. 

Program Evaluation 

Continue to engage in modifying the program’s design and delivery based on feedback 
from participants and Lab and other federal facilities staff. Evaluate the Institute’s 
effects on participants’ knowledge, attitude, and classroom practice. 
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SUMMER INSTITUTES FOR EDUCATORS 

This report summarizes the findings and recommendations developed from the 
evaluation of the 1993 FCCSET Summer Institutes. It is organized in three parts: 

Part I - The evaluation of the 1993 Summer Institutes; Part n - The evaluation of the 
1993 Summer Institutes Follow-up Activities; and Part in - The Recommendations 
resulting from both of these studies. 

Parts I and n each discuss the background, data collection and evaluation, and 
findings. 

PART I — EVALUATION OF 1993 SUMMER INSTITUTES 

BACKGROUND 

Part I of this report draws together the evaluations of 15 Summer Institutes for 
teachers held in June, July, and August of 1993 under guidelines and funding made 
available through the Federal Coordinating Council for Science, Engineering and 
Technology (FCCSET), now the National Science and Technology Council (NSTC). 
The Institutes were coordinated by the US Department of Energy (DOE). This study 
was sponsored by the National Science Foundation (NSF) and designed and monitored 
by the Dissemination and Evaluation Working Group for the NSTC and carried out by 
the National Center for Improving Science Education (NCISE) of The NETWORK, 
Inc. 

The Summer Teacher Enhancement Programs (STEP), known as Summer Institutes, 
provided opportunities for teachers from around the country to study at federal 
facilities. They spent four weeks in hands-on education programs in such varied areas 
as: environmental and ecological studies, material science, space research, and 
alternative energy and conservation. These Institutes drew on the wealth of scientific, 
technological, and human resources at host facilities across the nation and offered 
teachers the opportunity to deepen their knowledge in specific disciplinary areas. 

The goal of the 1993 Summer Institutes was to improve middle school/junior high 
and/or high school science learning by enhancing the science knowledge of teachers, 
already teaching science, through in-depth training. The Summer Institutes provided 
professional development for 560 teachers from 22 states and the District of Columbia 
in DOE Laboratories and other federal facilities throughout the country (see 
Appendix A). 

Different STEP program designs addressed different needs of teachers; some aimed to 
give teachers who did not have a strong science background the exposure that helped 
improve their ability and confidence to teach innovative, effective science courses. 
Other STEP programs addressed teachers who were comfortable teaching science in 
high school. These latter programs assumed a strong science background and sought 
to enhance teachers’ knowledge of the latest advances in research. 
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Additionally, some STEP Institutes provided teachers an opportunity to incorporate 
new information and teaching strategies into curricula and materials that helped them 
model the processes of technological innovation coupled with scientific discovery. 
Participants interacted with scientific and technical professionals, in laboratories, 
lectures, and in designing and conducting research experiments. In many cases these 
STEP experiences were the first opportunity participating teachers had to experience 
science and technology with cutting-edge scientists. Examples included: 

• building a cosmic ray telescope (at DOE’s Superconducting Supercollider 
Laboratory) 

• conducting research on magnetic fields present in a test bed and using a 
scanning electron microscope (at DOE’s Continuous Electron Beam 
Accelerator Facility) 

• tree coring and preparing the cores for research (at the Department of the 
Interior’s National Wetlands Research Center). 

Almost all of the Summer Institutes also drew upon additional presenters from such 
places as: institutions of higher education; varied research facilities; science and 
technology museums; and other resource centers. 

Please note: For the purposes of this report, the authors do not identify 
specific sites, participants, or staff members. The intent of the report is to 
reflect upon the Institutes as a whole, not as separate entities. 

Program Administration 

Requests for Proposals 

In December 1992, staff personnel from various federal Laboratories met to discuss 
the new FCCSET Initiative that had as its purpose: 

to expand content knowledge and enhance pedagogical skills of teachers 
through participation in Summer Institutes. 

They were invited to submit proposals. The emphasis was clearly on summer 
programs and on a 20-day (four week) Summer Institute as the delivery system. 
Follow-up was called for, but the idea of recruiting teams or of testing various 
delivery systems to support innovation in middle/junior high and high school 
classrooms was not mentioned in the RFP. Ways to link the goals and objectives of 
the Institute to reform efforts were addressed in the RFP, using examples such as the 
NSF’s Statewide Systemic Initiatives. Federal Laboratories and others applying for 
the grants were asked to state how school systems would support teachers 
participating in the Institute, but there were no apparent requirements that the school 
districts make significant contributions to this follow-up support (see Appendix B). 
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Institutes Selected for Awards 

Funding was awarded to 15 programs. Of the 15 programs, 14 were effectively first 
year projects and, therefore, under development as they took place. Even though 
some of the projects grew from earlier professional development efforts, they can be 
studied as first year efforts because they had to be adapted to fit the characteristic 
guidelines as delineated in the letter accompanying the Application Form (see 
Appendix B). For example: 

• Marshall Space Flight Center (Huntsville, AL) had run eight-week sessions, 
but never four-week events. 

• The Smithsonian (Washington, DC) substantially revamped their Institute as 
did the NASA Jet Propulsion Laboratory (Pasadena, CA). 

• Continuous Electron Beam Accelerator Facility (Newport News, VA) 
restructured an earlier plan for an Institute. 

Of the 15 projects, only one ran a program that they had offered several times before, 
with the same materials, time line, and overall structure. 

Sites and Institutes 

A variety of federal facilities were involved in this process in 1993, including: 

• Argonne National Laboratory (DOE), Argonne, Illinois - "FCCSET Summer 
Teacher Enhancement Institute" 

• Lawrence Berkeley Laboratory (DOE), Berkeley, California - "Science and 
Society Teacher Institute" 

• Clemson University and Pee Dee Research and Education Center (Agricultural 
Experiment Station, US Department of Agriculture), Florence, South Carolina 
- "PEAK (Professional Educators Access to Knowledge) Institute for Science 
in Agriculture" 

• Continuous Electron Beam Accelerator Facility (DOE), Newport News, 
Virginia - "Summer Institute for Teacher Enhancement" 

• Environmental Protection Agency and Miami University, Cincinnati, Ohio - 
"Hands-On Environmental Science" 

• NASA Jet Propulsion Laboratories, Pasadena, California - "Teacher 
Enhancement Program" 

• NASA Marshall Space Flight Center, Huntsville, Alabama - "Summer Teacher 
Enrichment Program" 
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• NOAA Laboratory (Department of Commerce), Boulder, Colorado - "NOAA 
(National Oceanographic and Atmospheric Administration) Summer Science 
Institute" 

• National Wetlands Research Center (US Fish and Wildlife Service, 

Department of the Interior) and University of Southwestern Louisiana, 
Lafayette, Louisiana - "Microscopy of the Wetlands" 

• Oak Ridge Laboratory (DOE), Oak Ridge, Tennessee - "Restoring Our Waters 
Teacher Institute" 

• Pacific Northwest Laboratory (DOE), Richland, Washington - "National 
Teacher Institute in Materials Science and Technology" 

• Sandia National Laboratory (DOE), Albuquerque, New Mexico - "GANAS" 
(Gaining Access to Natural Abilities in Science) 

• Sandia National Laboratory (DOE), Livermore, California - "SUPER" 

(Science Understanding Promotes Environmental Responsibility) 

• Smithsonian Institution, Washington, DC - "Natural Science Institute for 
Teachers of Minority Students" 

• Superconducting Supercollider Laboratory (DOE), Dallas, Texas - "Summer 
Institute for Physics and Physical Science" 

A distribution of Institutes by agency appears in Figure 1 . 



Figure 1: Distribution of Institutes by Agency 
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The fact that eight of the 15 funded sites were DOE sites reflects the fact that the 
majority of the proposals in response to the RFP were from DOE sites as well as the 
fact that the DOE Labs best met the RFP requirements based upon a panel review by 
experts in the area of teacher development. 
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Announcement of grant awards to the Labs and facilities was late; this foreshortened 
their recruiting resulting in some projects running with an enrollment under their 
authorized 50 participants. 

The Institutes all recruited public school teachers through announcements, fliers, and 
word of mouth in the spring of 1993. They were offered a summer experience of 
three or four weeks duration. Days were full, eight hours or more, with some 
evening sessions as well. Most Institutes met at federal facilities using 
classroom/conference spaces. Several Institutes made residential arrangements for at 
least some participants, while at others, teachers commuted or made their own 
housing arrangements. 

Budget Summary 

The grant awards for all 15 of the sites totaled $3,387,934. The smallest grant was 
$54,000 and the largest was $307,000. Stipends paid to teachers during the Summer 
Institutes exceeded $650,000. In virtually all cases, the budgets included funding for 
travel and other costs related to follow-up. Therefore, the funds expended per 
participant provided for some professional development beyond the four week 
Summer Institutes (see Figure 2). 

While original projections had hoped for a maximum attendance of 750 teachers, the 
actual attendance was 560 teachers at the 15 sites. This resulted in an average cost 
per teacher of $6,050. Had it been possible to fully enroll all Institutes, the cost per 
teacher would have dropped to $4,517. 

This figure reflects only the site costs for provision of the Institute and follow-up. It 
does not take into account in-kind contributions by the individual federal facility or 
the administrative costs incurred by DOE or the costs of evaluation. These figures 
are exclusive of the DOE administrative overhead costs which were not contained in 
the site-by-site data. 



Figure 2. Number of Teachers and Amount of Spending 
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DATA COLLECTION AND EVALUATION PROCESS 
The Template for Teacher Development 

To profile the 15 Summer Institutes a "Template for Teacher Development Programs" 
(see Appendix C) was used by project staff and site visitors. The template was 
developed by the National Center for Improving Science Education (NCISE), in 
collaboration with staff from various DOE Labs. Data from the template documents 
were coded and analyzed, along with other documentation, to prepare this report. 

The guidelines for use of the template ( Profiling Teacher Development Programs: An 
Approach to Formative Evaluation. NCISE 1993, 4) state: 

The purpose of profiling is to get a picture of what a program intends 
to do and what it actually does, in a way that enables comparison to 
best practice. The primary tool is the program template, which depicts 
the components of best practice (derived from both research and 
experience) and allows for description of a particular program in terms 
of those components. 

A completed template allows program managers to address the 
following questions: 

1. What is best practice in teacher development programs? 

2. To what extent is our program designed to reflect best practice? 

3. To what extent does our program actually reflect best practice? 

4. To what extent is the program’s design actually carried out? 

5. Where are the gaps? What can be improved? 

Working on behalf of the National Center for Improving Science Education, nine site 
visitors made trips to the Institutes (see Appendix D). Each Institute hosted at least 
one visitor (some hosted two) for approximately two days, typically during the last 
week of the Institute (or within the last 30% of the contact hours of the program). 

In terms of direct observation, these site visitors saw only the summer portion of the 
program. They were able to learn about planning for a variety of follow-up 
activities, but they did not directly observe any follow-up as it had not yet occurred. 

In Part I of this report, all comments about follow-up relate to planning of the follow- 
up and not to direct observation of those later activities. Part n addresses the 
implementation of the follow-up. 

Data Collection and Tabulation Using the Template 

Institute directors received training in the application of this template during the 
spring of 1993. They recorded elements of their program design in a category called 
"Intended" which paralleled a list of specific "Components of Effective Practice ," 
derived from research, literature, and best practice (see Appendix C). The site visitor, 
then entered observations for each of the intended items under a parallel column 
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labeled "Actual." These site visitors, using a combination of observation, informal 
discussion, and formal interviews with participants and staff were able to capture a 
"snapshot" of the state of the Summer Institute at the time the visit took place. 

Project directors also supplied additional background data through a "Descriptive and 
Context Information Sheet" (see Appendix C, FCCSET.TEM-9). 

Data Collection and Tabulation Using Pre- and Post-Surveys 

Further data were gathered through a "Pre-Participant Information Sheet," hereafter 
referred to as Pre-Survey, and a "Program Evaluation Form," hereafter referred to 
as Post-Survey, in which several of the same questions were asked, allowing for 
comparison and evaluation across the full breadth of each of the Summer Institutes 
(see Appendix E and F). Response rate to the Surveys was close to 60% 

FINDINGS USING THE TEMPLATE 

Findings from the Summer Institutes evaluation study are organized by the seven 
components of effective practice found on "The Template for Teacher Development. " 
These are: 

• Program Administration 

• Promoting a New Vision of Classroom Teaching and Learning 

• Teacher Development Program Activities 

• Uniqueness of the DOE Laboratories and Other Federal Facilities 

• Follow-Up 

• Teacher Leadership and Responsibility 

• Program Evaluation 
Component 1 — Program Administration 



This component includes the clarity of program goals, the recruitment process; : 
pre-program interaction and the inclusion of "teachers, scientists, educators and 
administrators in pjogramdesigm^ 



For each of the Institutes, the "Intended" column expanded upon the specifics that the 
program staff planned and carried out for each of the 1 1 identified elements of 
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program administration. In virtually every case, the site visitors were able to confirm 
whether these items had been addressed and successfully carried out before and 
during the Institute. In every Institute, limited recruiting time made the pre-program 
efforts less substantial and less successful than the Institute staffs had hoped for. 

Clear program goals were reported by site visitors at all but one Institute and a 
majority of participants at each Institute indicated their understanding of the goals. 
However, at every Institute there were some people who were not clear on the 
program goals or who felt there was some confusion in this area. Data from pre- and 
post-surveys indicate a need for improvement of advance communication prior to the 
start of (or enrollment in) the Institutes. Some of this can clearly be attributed to the 
first-year nature of these projects. 

Internal administration 

The site visitors found that the internal program administration of the Institutes was 
well organized. All of the Institutes found ways to involve their own program staffs 
and some Lab scientists in program design. Staff at many Institutes met on a regular 
basis during the three to four weeks of the sessions, and followed up on coordination 
details including outside presenters, resource materials, and the mentor relationships 
with scientists and other researchers. Few Institutes included teachers and school 
administrators in this design process. 

Inclusion of school administrators 

Four Institutes actively worked to involve school administrators for at least a modest 
portion of the Institute. One Institute was successful in getting an administrator to 
serve on each team. Aside from the program that included administrators on teams, 
the highest level of administrator participation in a portion of any Institute was 40% 
(in relation to the numbers of teachers enrolled in the Institute). Eleven Institutes did 
not link to administrators beyond obtaining written confirmation of teachers’ 
participation and an agreement to allow limited follow-up (see Appendix C, 
FCCSET.TEM-4). 

Selection process 

The time line between funding and start-up was very short, which meant the pool of 
applicants was quite small at many of the sites, resulting in low enrollments at some 
sites. The majority of participants were high school science teachers, with the next 
largest group being middle school/junior high teachers. Some Institutes included 
elementary teachers with one Institute having a majority of participants from 
elementary schools. 

Many of the Institutes had participants with varying skills. One Institute served 
senior high physics teachers and teachers of 9th grade physical science whose 
backgrounds in physics were quite different. There were suggestions that groups be 
separated by knowledge levels and that there be prerequisites for Institutes. Several 
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teachers commented that the selection process should also consider the leadership 
abilities and skills of the teachers in implementation of curriculum reform at the 
school district level. Teachers who are leaders would be able to contribute an 
understanding of such new efforts as: integrated and coordinated curricula at the high 
school level, problem solving skills at all levels, and how students can best move 
from concrete hands-on work to greater abstractions. 

Component 2 — Promoting a New Vision of Classroom Teaching and Learning 



"Promoting a New Vision ” refers tbgihe kind: of science program Icdxs want 
teachcmppmse wiihtf&r: students^ It relates to the vision of teaching and. :: 
learning embedded in the Institute; what the Institute hopes teachers will be able 
to accomplish in their classrooms; the on-going assessment of science learning; 
and the use of materials, strategies; and perspectives sensitive to di^ 
cultures, languages; genders, and learning styles. 



Many Institutes involved extensive content learning with detailed examination of the 
topics covered. It was less clear to site visitors that the Institutes promoted new 
approaches to student learning such that they would be better able to solve problems 
and develop habits of inquiry. Several site visitors repeated the observation that, "a 
new vision of the school and classroom generally is not emphasized. " Very few 
Institute instructors had experience in schools or were knowledgeable about the 
current research on learning and methods of science instruction. Therefore, the 
Institutes tended to emphasize science content, often in relatively narrow fields. 

While the exposure to such content was either a new or enriching experience for the 
participating teachers, they were not consistently offered a means to translate it to 
effective classroom practice. This is a significant issue that must be addressed by 
Summer Institutes in the future. 

One site visitor observed that a Summer Institute: 

offered extensive new content aimed at supporting high school physics 
teachers and some physical science teachers. It did not appear to focus 
intensively on improved pedagogy, a vision of the classroom, or on 
overall curriculum reform in science education. There was limited 
focus on assessment as it relates to schools. 

Some observers were concerned that the link to the classroom was not strong enough 
and doubted whether the substantial curricular issues inherent in offering such 
Institutes were adequately addressed by Institute staff and/or participating teachers. 

There was some confusion on the part of some Summer Institute staff about their 
vision for a science classroom. For example, support for the pedagogical concept of 
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"less is more" was often mentioned at the same time that there was a stress on 
covering more material in the curriculum. 

On the other hand, some observers saw significant attention to classroom issues in at 
least four of the sites. One site visitor wrote: 

There was clearly a vision of the classroom as one where students are 
active, hands-on/ minds-on learners and where learning is 
interdisciplinary in nature, where many aspects of science come 
together. 

Component 3 — Teacher Development Program Activities 



This section of the template examines how teachers are helped to develop new 
strategies , skills, and knowledge. It. includes examination of appropriateness of 
activities for adult learners, immersion in the scientific process, and inclusion off&- 
the use of tools and methods of scientists. Important elements include strategies . €[ ' 
that are transferable to the classroom, collaborative and cooperative learning, 
and the idea that teachers should be able "to plan for use of new knowledge and 
skills in their own classrooms with their own curriculum. tiiiS |:|i 



While every Summer Institute allowed a certain amount of these types of activities to 
take place, at least 10 of the 15 used substantial amounts of time for lectures and 
other large group presentations with limited input, problem solving opportunities or 
debate on the part of the teachers. Several observers noted the lack of opportunities 
for practice teaching with peer or instructor feedback. However, some field 
experiences were extremely interactive, encouraged problem solving, and allowed for 
excellent interchange between teachers and scientists. 

The comments of two site visitors give examples of their observations concerning this 
component: 

The process of investigation as a way of building scientific knowledge 
is under emphasized and often the field activities that teachers engage 
in (meant for them to adapt for their students) view science as a body 
of knowledge to be learned, a set of rules to apply. This is observed in 
teachers’ lesson plans and demonstrations. Students learn vocabulary 
first (abstract) before investigating or experimenting (concrete); there is 
emphasis on labeling or naming; there are more answers than 
questions; the big ideas or concepts are lost in the focus on facts. 

Participants were surprised by the interdisciplinary nature of science 
investigations, and noted their concerns with their own discipline-bound 
approaches in the schools. This richness of resources, however, also 
became a weak point. Many participants noted that the Institute 
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attempted to cover too much. They recommended that selected 
scientists, who could effectively model science teaching as it should be 
practiced and taught, participate in the future and that options be 
provided so that participants might select certain areas for more in- 
depth study. Moreover, they wanted more opportunities to process the 
information given to them by the scientists, and wanted to shadow some 
scientists in their Labs to better see how science is really carried out. 

The incongruity between instructional format and design of the Summer Institutes and 
the larger issue of reform of science education on the elementary, middle/junior high 
and high school levels was noted by every site visitor at virtually every one of the 
programs. Some of this was due to the planners’ limited experience with reform of 
science education on elementary/secondary levels and some to the limitations implicit 
in any Summer Institute which cannot do everything for all people at once. 

Component 4 — Uniqueness of the DOE Laboratories and Other Federal 
Facilities 



This section is designed to capture how the Jmtitidesttdce advantage of unique 
laboratory resources and missions, including scientists and technicians with |||| 
whom teachers might be able to interact directly: and who \wuldserve dsovle 
models. -t'-or .. ' ^ 



The " Intended " columns of the completed templates reveal that program staff at many 
sites anticipated that scientists and technicians would participate in program design, 
the development of the scientific/technical content, and collaborate with teachers to 
solve real or simulated problems using the equipment of the Labs and facilities. 

Institute staff in several locations commented that the limited pre-program time made 
lengthy collaboration with scientists and technicians difficult in the planning process. 

In view of that, many of the sites did well to involve the number of scientists they 
did. Site visits indicated that a range of three to eleven scientists were involved in 
each Institute. Establishing activities that the teachers could learn from and also 
understand was not an easy task at all the sites. That it was achieved to a modest or 
even notable degree at many of the sites was a substantial success. 

Role models in science and mathematics 

The issue of role models was not one that all the Labs could do anything about in the 
short term. Women and minorities were generally present in low numbers at the 
Labs and other sites. In addition, many individuals were simply not available to work 
with participating teachers on short notice. There were, however, strong exceptions 
to this generalization and the Labs could certainly do more in subsequent years to 
further the goal of greater diversity in role models. 
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Summer Institutes that offered a variety of interactive projects for teachers provided a 
substantial degree of choice to participants. These sites also provided for a degree of 
dialogue between participants working on different research projects. At least three 
Summer Institutes had as many as seven or eight different investigations in which 
teams of teachers participated with a mentor. In some, the teachers were fully 
engaged, actually working on a meaningful research project. In others, the teachers 
had to be more or less observers. 

The contributions of the scientists were significant and their responses to the 
participants make a major case for the use of these federal facilities in the future. 

One group of scientists, when asked if they would encourage such an Institute next 
summer, responded: 

"expand the project. " "Do it again. " "It isn ’t too great a demand to 
make on us as scientists. " 

Particularly valuable relationships developed between teachers and scientists/ 
technicians in those Institutes where an individual served as a strong liaison between 
the Institute staff and the scientific community. 

Content-rich environments 

The uniqueness of the laboratory environments as a place for professional 
development of teachers explains some of the intensely content rich aspects of the 
Institute. Each of the sites had resources, both human and physical, which were 
astounding to most of the teacher participants. 

Striking a balance' between the access to content knowledge and the application of 
frontier science to instructional change and curriculum improvement in schools is a 
massive undertaking. The problem of helping the instructional staff at these Institutes 
to be aware of new developments in and approaches to classroom instruction of 
students is a serious long-term issue and not one that can be coped with in a few short 
preliminary "staff meetings" prior to a single Summer Institute. 

Component 5 — Follow-Up 



The follow-up component of the template focuses on encouraging andireirfptdihg 
the use of new knowledge and skills in the participating teachers ' classrooms. % 



Plans for follow-up and early activities were noted by the NCISE study observers 
during their initial Institute visits. Details on these and the actual extended follow-up 
are found in Part II of this report. 
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Component 6 — Teacher Leadership and Responsibility 



This section of the template involves looking at the different ways teachers could l, 
share responsibility for aspects of the Institute ’s program, such as developjftftft, 
delivery , and implementation and explores the degree to which teachers could 
have input into decisions about the content, process, implementation and 
evaluation of their learning experiences: ^isisecnpn of the template also 
includes examination of teachers’ roles as leaders in their schools after their 
Institute experiences, as well as the long-term commitment and support from the 
laboratory to their schools and communities, y 



Once again, the limited time for planning made involvement of teacher participants 
prior to the start of the Institutes difficult. Most observers did not report significant 
attention to teacher leadership through the course of the Institute. Several Institutes 
did have small groups of teachers involved in the planning process (one to three 
teachers). At least two Institutes involved small groups of teachers in limited 
preliminary planning. 

Many teachers felt a responsibility to share the ideas they had been exposed to. That 
is, they indicated a level of personal responsibility for conducting in-service at their 
own school sites. Reported in the words of one site visitor: 

Teachers did not feel that they took on leadership responsibilities 
during the Institute and, in fact, felt somewhat helpless to influence the 
course of events. On the other hand, teachers felt they had the 
responsibility for spreading their learnings back home once the Institute 
was over; they planned variously to do in-services, have demonstration 
classrooms, and present their learnings to others. 

Several site visitors pointed to opportunities that appeared to be available for teachers 
to exercise additional responsibility during the follow-up process, but this could not 
be substantiated at the date of the summer visits. Again, this depends on the 
organization of the schools themselves and the degree to which teachers are allowed 
or encouraged to assume responsibility. Another site visitor wrote: 

Teachers were consulted throughout the Institute about how it was 
progressing and they were given opportunities to be leaders within their 
teams or sub-groups. It is difficult to judge how well this will translate 
to a school setting without greater support for leadership within the 
school itself. Teachers expressed a readiness to continue in their 
contacts with others/peers about science in the future. 
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Most Institutes allowed time for the preparation of plans for later in-service work by 
teachers. However, site visitors noted this was often done without the feeling that the 
teachers had clear administrative support or a firm knowledge of how or when the 
actual in-service could be scheduled. 

In large measure, this important component of the template cannot be fully tested 
without additional evaluation of participants in the first year and further attention to 
this topic in subsequent years of Summer Institutes run by these federal Laboratories 
and other facilities. 

Component 7 — Program Evaluation 



This section focuses on momtotihgiparticipaht satisfaction during the program^ 
prifgoihg adaptation, and both formative ahd Summative evaluati processes of 
the Summer Institute staff during the general time frame of the Summer Institute . 



In general, the Summer Institutes gathered data from participants and carefully 
reviewed the needs of teachers throughout the duration of the Institutes. Teachers at 
several sites commented that modifications were made based upon the evaluation 
process. Journals were used as an interactive tool in at least four of the Summer 
Institutes, with ideas being shared among participants and with program leaders. 

Short questionnaires were also used at several sites on a daily or twice-weekly basis 
to gain information and feedback about presenters and the degrees to which a portion 
of the program met overall goals. Given the complexity of scheduling and the variety 
of presenters that had been arranged at many of the Institutes, the program staffs 
appear to have done a remarkable job in making modifications in these first-year 
sessions. 
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FINDINGS USING THE PRE- AND POST-SURVEY 

Following is a summary of the findings related to participant expectations, level of 

satisfaction and comfort level with subject matter, and instructional techniques. 2 

There were 328 responses in the Pre-Survey and 338 in the Post-Survey. 

Expectations 

Question #6 in the Pre-Survey (Figure 3) and question #3 in the Post-Survey 
(Figure 4) were compared to obtain information regarding the participants’ 
expectations as compared to their actual experiences. On a scale of 1-3 the 
mean distribution ranged from 2. 7-2. 9, indicating participants had uniformly 
high expectations with regards to learning content, laboratory skills, 
applications, pedagogy, and activities that could be transferred to their own 
classroom setting. For the most part their expectations were met; the lowest 
score (3.3) referred to their having learned laboratory skills that they could 
teach their students. 



Figure 3 

Pre-Survey #6 — Indicate your expectations in regard to the program (either during the program or as a result of 
your participation). 




Expectations (Pre) Summary of Responses 
(Figures in Columns 1-3 are in Percentages) ' 




§§§|| 

v ;. 

expect this 
will occur 


flps . 

Sure this :: 
Witt occur 


■ :3 *: 

I expect 
thiswiH: 
occur 


; . '-Mean : xx 




a. 


observe scientific research in the laboratory 


4.8 


16.6 


78.6 


2.7 


.54 


b. 


increase my science/ mathematic s/technology content 
knowledge 


1.0 


7.2 


91.8 


2.9 


.32 


c. 


increase my knowledge of applications in 
sc ienc e/ mathematic s 


1.0 


7.3 


91.7 


2.9 


.32 


d. 


gain new perspectives on how science/mathematics/ 
technology should best be taught 


1.7 


10.8 


87.5 


2.9 


.40 


e. 


leam about activities I can use in my classroom 


0.7 


10.0 


89.3 


2.9 


.34 


f. 


develop activities I can use in my classroom 


0.7 


13.4 


85.9 


2.8 


.38 


g- 


leam about how to use specific equipment and 
technologies in my classroom 


2.1 


20.4 


77.5 


2.7 


.47 


h. 


leam laboratory skills that I can teach to my students 


4.2 


20.1 


75.7 


2.7 


.54 



2 Total responses reflect input from ten out of fifteen Labs/facilities: two Labs did not 
submit post surveys, one Lab did not submit any pre or post survey results, one Lab 
misinterpreted questions and one Lab did not submit pre-survey results. 
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Figure 4 

Post-Survey #3 — Indicate the extent to which you agree with each of the following statements about what occurred 
during your participation in the program. 






What Learned (Post) Summary of Responses 
(Figures in Columns 1-4 are in Percentages) 






1 4 

Not at all To a great extent 


Mean 


SD 


a. 


I increased my science/mathematics/technology 
content knowledge 


0 


8.8 


32.0 


59.2 


3.5 


.65 


b. 


I increased my knowledge of applications in 
science/mathematics 


0 


3.8 


32.6 


63.6 


3.6 


.56 


c. 


I gained new perspectives on how science/ 
mathematics/ technology should be best taught 


0.6 


4.7 


26.9 


67.8 


3.6 


.60 


d. 


I learned about activities I can use in my classroom 


0 


4.2 


22.2 


73.6 


3.7 


.55 


e. 


I learned laboratory skills that I can teach to my 
students 


2.7 


13.3 


31.4 


52.6 


3.3 


.81 
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Level of Satisfaction 

Participants rated their level of satisfaction with the program by completing 
question #4 on the Post-Survey (Figure 5). On a settle of 1-5, the mean 
ranged from 3.9 to 4.6 with advance communication receiving the 3.9 and 
orientation 4.0. This reflects back to the lateness of award notification to the 
programs. Overall, the average mean was 4.33 indicating a level of 
satisfaction of very good. 



Figure 5 

Post-Survey #4 — Rate the following aspects of the program. 

(Figures in Columns 1-5 are in Percentages) 




f‘ ' 


;:E1 

Poor 


2 • 
Fair 


3||i 

Good 


4 

Very Good 


Excellent 


Mean 


■ vSDrv 


a. 


Program administration 


0.9 


3.3 


14.0 


35.8 


46.0 


4.2 


.87 


b. 


Advance communication 


3.3 


9.5 


18.8 


29.4 


39.0 


3.9 


1.12 


c. 


Orientation 


0.9 


7.2 


18.6 


38.0 


35.3 


4.0 


.95 


d. 


Availability of resources 


0 


3.0 


11.9 


33.9 


51.2 


4.3 


.80 


e. 


Assistance provided by program 
staff 


0 


0.6 


4.4 


27.7 


67.3 


4.6 


.58 


f. 


Workshop leaders 


0.3 


0.9 


7.6 


34.7 


56.5 


4.5 


.70 


g- 


Interactions with other teachers 


0.3 


1.2 


8.0 


24.9 


64.7 


4.5 


.73 


h. 


Interactions with 
sc ientists/technic ians 


0.3 


1.8 


10.4 


30.1 


57.4 


4.4 


.77 


i. 


Receiving advice and support for 
sharing experience 


0.6 


1.8 


9.3 


32.3 


56.0 


4.4 


.78 


j- 


Receiving support for extending 
experience to the classroom 


0.3 


1.8 


9.0 


29.7 


59.2 


4.5 


.76 
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Comfort Level 

Comparisons of participants comfort level with content and transfer to the 
classroom were obtained by their responses to question #5 on the Pre-Survey and 
to question #6 on the Post-Survey (see Figure 6). On the Pre-Survey respondents 
indicated they had a fairly good comfort level with their level of knowledge and 
ability to transfer to the classroom and in the Post-Survey as well. Although there 
were no statistically significant differences, there was a slight change towards 
greater comfort with the greatest changes noted in their increased confidence in 
their ability to inform their students of various science/mathematics and 
technology career opportunities and their current level of science, mathematics and 
technology knowledge. 



Figure 6 

Pre-Survey #5 and Post-Survey #6 — Indicate the 
degree to which you agree or disagree with each 
of the following statements. 


Figure 6 

Figures in Columns 1-4 are in Percentages 
Comfort Level (Pre) — Summary of Responses (Shad< 
Comfort Level (Post) — Summary of Responses (Unsha 


id) 

ded) 




X 

: f : 

Agree 


Iflll 

lljAgree 


$ 

Disagree 


-.4 

IStroogb' 

'■Disagree::: 




SD 




I am comfortable with my current level of 
science/mathematics/technology knowledge 




45.8 


i^§n 


III!! 




.82 


a. 


19.2 


52.0 


23.1 


5.7 


2.1 


.79 


b. 


1 feel comfortable teaching 


lllllll 


514 




3.7 


1.8 


.76 


science/mathematics/technology 


45.2 


47.0 


5.1 


2.7 


1.6 


.70 


c. 


1 feel comfortable managing a class of students 


. . . 49.4 ‘ 


|®||1 


G is 


11! 2.7 


1.6 


.72 


who are doing hands-on activities 


62.7 


32.5 


2.4 


2.4 


1.4 


.66 




1 feel comfortable demonstrating 


;•! -: : 4p.3X 


51.0 


§|§ 


mam 


llllll 




d. 


science/mathematics/technology principles to 
my students 


56.9 


36.4 


4.8 


1.9 


1.5 


.67 


e. 


1 feel confident in my ability to discuss 
science/mathematics/technology applications 
with my students 


■ni 


!!§ |§|§! 


13.9 


lllllll 


• 1.8 


PI! 


44.3 


47.3 


6.0 


2.4 


1.7 


.70 


r 


1 feel confident in my ability to inform my 


23* 




lllllli 


IS 4.0 


§;; 24 


1 ill 


t. 


students of various science/mathematics/ 
technology career opportunities 


35.3 


51.8 


10.2 


2.7 


1.8 


.72 




1 feel confident in my ability to help my 
students answer their own questions 




: . 55.5 


.. 9.3 




111111:1 


: M 


g- 


45.9 


47.4 


4.5 


2.2 


1.6 


.67 


h. 


1 feel confident in my ability to 


flf . 304 


52.8 




2.5 




.73 { 




supervise my students’ research projects 


46.1 


43.4 


8.0 


2.5 


1.7 


.73 
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Overall, the Institutes received substantial credit and applause from 
participants. A participant from Washington, DC may have summed up a 
substantial number of opinions by saying that one Summer Institute was 
designed for, "any teacher who was willing to open the door and go through 
it." 

SUMMARY OF FINDINGS 

Overall NCISE site visitors found the Institutes to be of very high quality. These 15 
Summer Institutes began a valuable process of exploring effective interactions between 
teachers and diverse federal facilities, all with research missions. During the four 
week programs, staff members were coping with modifications and improvements on 
an almost daily basis at most of the Institutes. 

Of the 15 projects, only one ran a program that they had offered several times before, 
with the same materials, time line, and overall structure. It is interesting to note that 
this Institute received praise as a coherent, teacher-centered program with significant 
science input, while also offering reflective time for groups and individuals to process 
the experience. 

One commonality that affected all the Institutes was the lateness of grant 
announcements, which made the recruiting process difficult because it was so 
foreshortened. This reduced the pool of available applicants and caused some of the 
projects to run with an enrollment under their authorized 50 participants. In general, 
the recruitment process did not allow for any selection of teachers. Virtually all who 
applied were accepted with many being notified late in the spring. An unknown 
number of teachers who were accepted had to decline because of late notification. 
Survey data indicated that advance communication, prior to the start of the Institutes, 
could be improved. 

Teachers appreciated the respect they experienced from scientists and staff and 
exposure to science, mathematics and technology professionals, and sites and 
equipment they had never encountered before. This level of exposure alone could 
have valuable long-term consequences for teacher motivation and interest, as well as 
knowledge base. 

Assuming funding in subsequent years, ways to strengthen both programmatic and 
pedagogical elements were identified by both the Summer Institutes staff and NCISE 
site visitors. These are reflected in the recommendations found in Part HI of this 
report. 
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PARTICIPANTS COMMENDATIONS IN REGARD TO 1993 SUMMER 
INSTITUTES 

Teacher participants indicated the following commendable activities in regard to their 
experiences in the Summer Institutes: 

• Expressed a great deal of enthusiasm for the Summer Institutes. Terms such 
as "invigorated, renewed, and validated" as a result of their experiences were 
heard many times. 

• Felt the high level of demand made on participants was uplifting and 
rewarding to them. 

• Praised the Institutes for their immediate response to teacher suggestions 
resulting in program adjustments and logistical changes. 

• Lauded the Institutes for their excellent organization. Attention to 
administrative details meant program considerations could take precedence. 

• Indicated they had a better understanding of the relationship of science to real 
world applications and had exposure to state of the art scientific developments. 

• Felt the Institutes provided them the opportunity to have personal interaction 
with and acknowledgement by scientists as well as providing opportunities for 
them to interact and exchange among themselves in a learning environment. 

• Expressed that the Institutes provided them the exposure and opportunities to 
learn together with teachers from different geographic areas and cultural 
backgrounds. 

• Obtained an increased awareness of local resources and increased confidence 
in doing "hands on" science as a result of their Institute experience. 

• Were reassured by the promise of follow-up provided to them. 
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PART II — EVALUATION OF 1993 SUMMER INSTITUTES 

FOLLOW-UP ACTIVITIES 



BACKGROUND 

The 15 Summer Institutes proposed specific follow-up procedures at the time the 1993 
proposals were written with most activities to take place during the 1993-1994 school 
year. Geographic distances between participants’ schools figured in the follow-up 
plans of nearly every 1993 Summer Institute site. The time and the cost involved in 
bringing people together for one or more follow-up sessions are considerable even for 
those 1993 Summer Institutes that recruited largely from their own states. 

Every Institute obtained signed agreements to allow teachers to participate in follow- 
up session(s) and administrators were to be included in follow-up sessions by at least 
four Institutes sites. During the year, all of the 1993 Summer Institutes carried out 
some form of follow-up. This part of the report examines what these follow-up efforts 
were and how they were carried out. 

Many possible varieties of follow-up were noted, including additional training, on-site 
or telephone consultation, networking through newsletters or telecommunications, and 
the further training of local resource teachers or others who might provide on-going 
assistance to classrooms. 

Virtually all of the proposed activities called for group meetings of the participants on 
one or more occasions in the school year following the Institutes. For Institutes that 
planned to bring the group of participating teachers together, these meetings were 
usually between November 1993 and March 1994. 

All of the proposals anticipated the production of follow-up print media, either as 
material coming from teachers themselves or from the various projects. One site 
planned to edit and publish teacher-written activities and circulate this document to all 
participants. Three other Institutes developed handbooks to further support teachers 
in their classrooms. These handbooks ranged from curriculum and activity outlines to 
resource lists of people and institutions. 

Thirteen of the proposals discussed the anticipated use of telecommunications as a 
means to link teacher-to-teacher and to tie back to the Lab as a resource. At least 
three sites planned to loan equipment to schools, arranging for delivery and some 
additional support services. 
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DATA COLLECTION AND EVALUATION PROCESS 

During spring and summer 1994, NCISE collected data in several ways about follow- 
up efforts. These included: 

• Site visits to eight projects during scheduled follow-up sessions (see 
Appendix G). (Only eight of the fifteen projects had follow-up events 
scheduled during the period of the follow-up study.) Site visitors conducted 
formal interviews with teachers and staff, and observed program elements; 
they also discussed and observed other follow-up events and services. Several 
quotations from these reports appear in the following pages, as part of the 
documentation of follow-up activities (see Appendix H and I for site visitor 
guidelines and sample follow-up questions). 

• Phone interviews with directors at all 15 sites during which they were 
encouraged to discuss actual follow-up services, along with their assessment of 
the effect of follow-up. Many discussed problems they had encountered and 
their plans for future improvement. 

• Phone interviews with participants, usually reached at their homes, during 
which a similar series of questions were asked to ascertain their level of 
involvement in follow-up and its utility to each of them. In these interviews, 
participants’ responses included anecdotal comments in regard to their own 
involvement in specific Institutes. Although an effort was made to reach a 
randomly selected group of 100 participants, only 63 could be successfully 
contacted. 



A summary of data collection activities appears in Figure 7. 



Figure 7 


. Data Collection for Follow-up 
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Institute Director Interviews 










Eight sites 


All 15 sites 


63 teachers 






from all 15 sites 



Information from all of these data sources is drawn together in this narrative report, 
examining the impact of follow-up efforts after the 1993 Summer Institutes. By the 
time of the interviews for this follow-up study, virtually all of the proposed activities 
for 1993 participants had been completed. However, several Institutes indicated that 
they would continue to communicate with past participants, at least on the level of 
newsletters and releases of upcoming program information. 
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Limitations 

The data collection efforts netted information about elements of the follow-up at each 
site. Strict comparison of all the sites is not possible for every element of their 
follow-up programs, because of the diversity of the follow-up at the various sites. In 
many cases, the follow-up activities were limited in scope or served only a small 
percentage of the teachers originally enrolled in the program. 

Another consideration in examining follow-up from the 1993 Institutes is the degree to 
which staff changes at the DOE Labs and other federal sites may have affected 
follow-up. At four of the sites, follow-up was conducted largely by individuals who 
had not been involved in the original Institutes. 

FINDINGS REGARDING THE TYPES OF FOLLOW-UP 

Follow-up took many forms such as main events scheduled by Institutes, 
telecommunications, sharing of equipment and personal follow-up such as phone calls 
and site visits. 

Main Events 

Each project scheduled a "main event" at some time during the year. This was a 
gathering to which some or all of the participants were invited. Two projects had a 
very high rate of involvement in these events, but the average level of participation 
across all projects was approximately 45 % of those invited (this includes the noted 
high attendance at two projects). In teacher interviews, estimates of participation in 
these main event follow-up sessions were generally lower than the estimates provided 
by project directors. This may have been because the directors were reporting levels 
of attendance related to people who had said they would attend, not the full number of 
Summer Institute enrollees. 

The actual pedagogy of the "main follow-up events" differed a great deal from site to 
site ranging from no correlation to classroom practice to discussion of classroom 
applications and site visits to participants’ classrooms. Teachers expressed 
appreciation of the follow-up effort, but they also reported little direct involvement 
with the process. Three sites that were visited by NCISE staff highlighted teacher 
participant presentations at follow-up sessions, but only one had a session planned by 
a teacher participant. Of this session, a site visitor reported: 

Rather than starting with a lecture, the teacher leading this session gave 
a brief set of instructions so they could get on the university computer 
network and then encouraged her colleagues to "discover. " She also 
gave them a technique to use when they needed help that they could, in 
turn, use in their own classrooms as a management technique. Thus, 
the entire morning proved to be an effective model of how teachers 
could get started in their own classrooms. The other teachers received 
the session enthusiastically. 
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A number of sites used expert presenters for follow-up on specific topics. In many of 
these cases, the experts were scientists well-versed in their own fields, but with no 
experience in schools. Teachers who attended these sessions reported that there was 
little dialogue, and it was not clear what aspects of the presentation they could use in 
schools. 

Telecommunications 

Telecommunication using e-mail and modems was mentioned, at least as an option, in 
all original proposals. Reports from directors and participants indicates that this 
happened sporadically and only in some follow-up efforts. One Institute follow-up 
had ten participants linked by the Internet, a success rate of 22% for that project, and 
at least one site held an entire half-day follow-up session on telecommunications for 
educators. 

Information about e-mail, from teachers and others, suggests that many of the 
roadblocks occur at the school level. Equipment is not in place in many schools, or it 
is non-functional. The largest single issue beyond computer-software-modem 
concerns is the availability of dedicated phone lines. Many schools seem 
unwilling/unable to provide these connections even when the hardware and software 
problems are solved. A site visitor accompanying an Institute director and a senior 
scientist on a school visit reported: 

They also wanted to find out what resources the teacher had access to, 
specifically if she had a computer, modem, and phone line so that she 
could tap into the Lab’s telecommunications network. When they 
learned she did not have a modem or phone line, they tried to persuade 
the principal to provide the teacher with these resources and suggested 
methods for paying for them. 

In the latter example, the substantial costs of training the teacher and dedicating a 
portion of the Lab’s computer network to services to schools ran afoul of a school 
system that would not or could not complete its share of the bargain, agreed to when 
the teacher signed up for the Institute. However, other teachers mentioned that they 
knew that their school districts had signed letters of support for those participating in 
the Summer Institutes. They indicated they had used this fact to urge administrations 
to provide additional equipment, communications services, and other functions. 



Sharing of Equipment 

A further aspect of follow-up was the sharing of equipment on a loan basis or the gift 
of equipment to schools. 



Computers were transferred from at least seven Labs in this process to benefit 
teachers who had participated in the Summer Institutes. Laboratory equipment was 
provided by at least three Labs. 
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The most active loan programs included transportation of equipment. At least one 
Institute actively staffed the loan program, as reported in this site visitor’s words: 

The Institute saw its role with the teachers as a continuing one. Those 
teachers who participated in the summer of 1993 will be able to 
continue to borrow the equipment in the 1994-95 school year, provided 
that they take a brief in-service workshop reviewing the proper use of 
the equipment. To help teachers to use the equipment, the Institute 
gave the responsibility for distributing and monitoring the equipment to 
one staff member who was always available (when not transporting the 
equipment) to answer questions about its use. Almost all the teachers 
said they had called him and found his answers helpful and effective. 

In five of the 15 sites teachers reported difficulty operating loaned equipment. 

Personal Follow-up 

Another follow-up strategy used by the Summer Institutes, but mentioned only rarely 
in original proposals, consisted of informal telephone calls from Institute staff to 
teachers. Several projects indicated that they called participants (about scheduling or 
to check on progress). Project directors could not quantify the response rate 
precisely, but one indicated that they had reached "well over half' of the participants. 
However, in a number of cases, involving at least 11 of the sites, some teachers 
reported being contacted by phone as part of the follow-up effort while others (at the 
same site) reported no contact. Of participants interviewed by phone, 38% indicated 
that they participated in no face-to-face follow-up. 

Classroom or school visits were also employed by Lab staff to observe and support 
participating teachers. In some instances the visitors also met with the school 
principal. The visits were an opportunity for Institute staff to see what needs 
individual teachers had that they could assist with, assess how the teacher was 
incorporating the material learned over the summer, pass on information from other 
teachers, and keep the connection between the teachers and the Institute active. Some 
teachers described these visits as reinvigorating. 

Just as the Institutes themselves often delivered knowledge with limited attention to 
instructional techniques, it appeared from interviews that a significant majority of the 
face-to-face follow-up sessions were also content-rich but limited in pedagogy. 

Some teachers were able to bring their classes to the Lab site that offered their 
Summer Institute. Those who did were enthusiastic about the value of the trip and 
support of Lab staff. 
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FINDINGS REGARDING PLANNING OF FOLLOW-UP 

Where teachers played an active role in the planning of the follow-up process, there 
was a general enthusiasm for the activities. Although it was impossible for 
interviewers to ascertain this from all sites, it appears that at least six of the 15 sites 
directly involved teachers in planning follow-up sessions in situations where groups of 
teachers gathered for a full day or half-day. 

Overall, there were dramatic differences in the degree to which teachers were 
involved in the planning and interactions around the follow-up process. Examples 
drawn from two different site visits serve to highlight this. A site visitor who 
watched one follow-up event and who had a chance to talk to several teachers in the 
session commented: 

A major weakness in the follow-up activities is the dependence on 
scientists who may or may not have well developed teaching skills. 

The observed workshop consisted of a lecture about the nature of 
minerals and rocks and an overview of the materials in a packet of 
geology materials for classroom instruction that the teachers were to 
pilot. The geologist making the presentation admitted that he did not 
know much about teaching children and was unable to develop many 
teaching activities. Although participants did have an opportunity to 
ask questions, there was little opportunity for them to discuss how to 
present geology to students in general or how to use the packet of 
materials specifically. 

In contrast to this assessment of a follow-up session that had little immediate value to 
teachers, another site visitor reported on follow-up at an Institute site in which 
agricultural extension agents had been called upon to support teachers in the field: 

The initial organization of the Summer Institute supported the 
implementation of its teachings. Creating teams of teachers with an 
extension agent built in self-perpetuating follow-up support. The center 
did not have to monitor and guide the teams, as they were truly 
independent. Having the team create a plan before they left the 
Institute gave the teachers something specific to work for during the 
year. Certainly, not all plans developed the way the teachers had 
anticipated, but at least they had something to build upon. The 
extension agents helped the teachers make needed contacts and supplied 
technical advice. . . . 
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ADDITIONAL FINDINGS 

• Teachers from four of the 15 sites reported interactions between their Summer 
Institute and NSF funded "statewide systemic improvement" projects. 

• Teachers from at least nine of the 15 sites reported informal contact among 
participants during the following school year (beyond any Lab-initiated follow- 
up). 

• In four cases teachers reported unequal access to follow-up services. In some 
cases this was due to geography, where great distances made it difficult to 
have site visits. 

• Some sites tied follow-up activities to the stipend and others did not. Thus, 
some teachers chose not to participate in follow-up for which they felt they 
were not compensated. 

• Many teachers seemed to feel that follow-up did not carry the importance of 
the initial Summer Institute. This was due in part to the limited nature of 
follow-up and also due to the fact that extensive follow-up was not available to 
everyone, due to geography. 

• Some sites planned for on-going teams to work together in the schools. 
Interviewees suggested that this did not happen in any significant way. The 
teams did not have a place in the structure of the schools or school districts or 
did not have adequate time to meet. 

In various forms, school system roadblocks were reported by a significant number of 
Institute directors and in teacher telephone interviews. Several teachers in a follow-up 
session at another Institute reported that they: 

"...felt hostility from their administrators toward the type of activities they learned 

at the Institute. " 

One of these teachers noted that: 

"...his principal did not want the students doing hands-on activities, but learning 

the ‘scientific method’." 

Such confusion about instructional practice on the part of a school administrator can 
confound the efforts of able teachers and the best professional development Institutes. 
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SUMMARY OF FINDINGS 

Overall, interviewers noticed a sense of interest and collegiality among many teachers 
who participated in this study. They were pleased to talk about the Summer Institute 
experience and follow-up, even in cases where they had little or no contact with the 
Lab site after the Institute. All participants interviewed indicated that they had been 
invited to participate in some form of follow-up interaction. 

Just as the original Institutes differed in their content, process, and instructional 
techniques, the follow-up initiatives varied greatly. While the telephone interviews 
and site visits attempted to develop a unified view of the follow-up process, there was 
great variation in the specific events of each follow-up effort. 

Telephone interviews indicated there was a general sense that follow-up sessions 
addressed certain aspects of actual classroom implementation that had not been 
explored in the Summer Institutes themselves. 

Additionally, data gathered through telephone interviews and site visits showed that 
participants talked and shared informally with one another. These interactions may 
not have developed specific classroom applications, but they provided a rich source of 
ideas and inspiration, with teachers learning from each others’ experiences. Many 
teachers felt isolated in their schools, and these informal networks and follow-up 
sessions helped them make professional connections. 

Teachers who attended Institutes that incorporated follow-up as an integral part of 
their complete program tended to be more active in incorporating material learned 
during the summer than those teachers who attended Institutes that did not place a 
strong emphasis on follow-up. In four of the site visits, and in 15 % of all the teacher 
telephone interviews, participants identified the significance of the follow-up process 
and the enthusiasm it generated. 

Few follow-up sessions afforded hands-on opportunities or other events that might be 
defined as inquiry or constructivist learning. However, those participating teachers 
who experienced hands-on learning opportunities found them to be most useful. 
Generally, these were activities that were challenging to the adults and usable in many 
classroom situations. 

Follow-up raises important geographic questions, among them the service area of the 
individual project. Institutes that drew from a national or multi-state pool of teachers 
had fewer options in conducting follow-up than those programs serving the immediate 
region. One Summer Institute gave preferential follow-up treatment to in-state 
teachers, essentially as if the funding were state-specific although attendees came from 
several states. Other Institutes recruited only within their states at the beginning of 
the process. While there is an argument that serving teachers on a local/regional 
basis encourages follow-up communication, there is also a need to use federal dollars 
and services equitably, making services available to teachers throughout the U.S. 
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PART III — RECOMMENDATIONS 

PROGRAM ADMINISTRATION 

• Continue the process begun in 1993, drawing upon the resources of federal 
Laboratories and other facilities to improve learning in American schools. 

• Relate the funding cycle to the school year to allow for adequate recruiting, 
pre-program planning time, and decision-making time for teachers. 

• Determine the audience for a given Institute and recruit participants based on 
the Institute’s clearly established goals. This step will also clarify pre-program 

communications. 

• Explore multi-year contracts, allowing a Lab to engage in on-going 
professional development including follow-up and other forms of support 
which the Lab could plan for and carry out over a longer period. 

• Design the participant selection process to ensure selection of teachers with 
leadership abilities and skills who can transfer their experiences in the Summer 
Institutes to their educational system. 

• Examine the benefits and drawbacks of drawing teacher participants with the 
same knowledge base or grouping them at times during the Institute according 
to knowledge level. 

• Consider recruiting teams from schools or school districts, such that the 
members could support one another upon return to the classroom and could 
better develop effective in-service opportunities for their peers. 

• Increase diversity (both gender and ethnic) of participants and presenters, 
bringing both role models and potential mentors to the fore. 

• Recruit teams of participants from broader geographic areas. 

• Explore cost-saving options to reduce the cost per teacher. 

PROMOTING A NEW VISION OF CLASSROOM TEACHING AND 
LEARNING 

• Articulate and communicate to participants a vision of how learning and/or 
teaching of science can look in the classroom. 

• Examine emerging national standards for teaching and learning and other 
reform documents to help communicate the new vision and connect it to other 
reform initiatives. 
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• Involve a team of teachers in planning an Institute and its follow-up. These 
teachers should be chosen in part for their knowledge of and skill at 
instructional practice. 

• Work in advance with outside presenters to further their understanding of adult 
learners and common goals and methodologies, so there is consistency 
throughout the Institute. 

• Encourage a greater understanding and clarity of vision among Lab education 
staff and scientists of new research and practice in science education and 
cognition. Help them model the effective practices they hope teachers will use 
with their students. 

TEACHER DEVELOPMENT PROGRAM ACTIVITIES 

• Ensure that education staff working with participants model pedagogy 
reflecting the latest research on effective teaching. 

• Ensure that the relationship between instructional format and design of the 
Summer Institutes and the larger issue of science and mathematics education 
reform be clearly defined. 

• Arrange dialogue and visits between scientists and participating teachers, 
creating a stronger link between these two groups and encouraging better two- 
way communication. This can be done both face-to-face and through 
telecommunications over an extended period. 

• Provide concrete experiences early in any inquiry, building opportunities for 
problem solving and developing meaning from experience. This will model 
effective instructional approaches that encourage creativity and a spirit of 
cooperative inquiry among the teachers’ students. 

• Decrease lecture time and increase actual investigation and immersion time 
during the Institutes. This may necessitate smaller groups. 

• Focus on linking new knowledge being developed by participants to larger 
understanding of science. 

• Fund several models of professional development for experimentation. 

• Allow some choice by teachers of alternative experiences they may explore in 
conjunction with each Institute. This should be guided by research and 
practice relating to adult development. 

• Consider formats other than a 20-day Summer Institute. 
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UNIQUENESS OF THE DOE LABORATORIES AND OTHER FEDERAL 

FACILITIES 

• Consider how to make specific science research skills possessed by the staff in 
the Labs and other federal facilities available to science and mathematics 
teachers across the nation on a continuing basis. 

FOLLOW-UP 

• Focus the follow-up site visits on technical assistance for implementation. 
Sustaining knowledge and skills and helping teachers disseminate their 
learnings to their colleagues require a high level of on-going support 
throughout the year. 

• Plan follow-up that becomes part of a longer-range process and ensures a 
relationship between the teachers and the Labs and other federal facilities 
offering the Institutes. Rather than viewing the Institutes as a summer event 
with some type of check-in during the school year, educators should think of 
themselves as on-going participants, linked to scientific resources, but also 
sharing their own curricular expertise through a growing network of science 
education activists. 

• Obtain commitment to specific follow-up strategies early in the process from 
teachers and school administrators. If this means release time for travel and 
meetings, that should be arranged. If these strategies require a modem and 
telephone line, along with an e-mail system, that too should be arranged in 
whatever fashion is necessary to encourage the full participation of the 

teacher. 

• Involve teachers in follow-up planning and execution. For this to be 
successful, it would have to begin early in the schedule of each Summer 
Institute, perhaps with a core group of teachers. 

• Provide "hands-on" follow-up activities that could be easily transferred to the 
participating teachers’ classrooms. Most teachers said that they found hands- 
on learning opportunities to be the most useful. 

• Provide inclusive follow-up in all projects, so that no circumstance arises in 
which a teacher is left out of follow-up because of geographic, technological, 
or financial restrictions. 

• Provide functional equipment to participants if distance learning techniques are 
to be used. Many schools do not have the hardware, software, phone lines, 
and expertise to support telecommunication. 
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TEACHER LEADERSHIP AND RESPONSIBILITY 

• Increase the level of responsibility of all teachers in the planning and follow-up 
process. 

• Allow time for teachers to practice new units of instruction and to obtain 
feedback within the Institutes. 

PROGRAM EVALUATION 

• Continue soliciting data from teacher participants to modify present program 
administration, design, and delivery to best fit the participants’ needs. 

• Continue to obtain data from the Lab and federal facility staffs and use this 
information to modify the design and delivery of future programs. 

• Pursue further evaluation to ascertain the changes in teachers’ knowledge, 
attitude, and classroom practice as a result of their participation in the Summer 
Institute. 
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Sites and Projects Listed by Department or Agency 

In order to emphasize the richness of resources and the variety of projects in which 
teachers engaged, we have reorganized the list of Summer Institutes here, breaking it 
down by federal agency and highlighting some of the topics investigated under the 
leadership of scientists and technicians at the site: 

Department of Energy 

PACIFIC NORTHWEST LABORATORY, Richland, Washington 

"National Teacher Institute in Materials Science and Technology" 

SANDIA NATIONAL LABORATORY, Albuquerque, New Mexico 
"GAN AS" (Gaining Access to Natural Abilities in Science) 

SANDIA NATIONAL LABORATORY, Livermore, California 

"SUPER" (Science Understanding Promotes Environmental Responsibility) 

ARGONNE NATIONAL LABORATORY, Argonne, Illinois 
"FCCSET Summer Teacher Enhancement Institute" 

SUPERCONDUCTING SUPERCOLLIDER LABORATORY, Dallas, Texas 
"Summer Institute for Physics and Physical Science" 

OAK RIDGE LABORATORY, Oak Ridge, Tennessee 
"Restoring Our Waters Teacher Institute" 

LAWRENCE BERKELEY LABORATORY, Berkeley, California 
"Science and Society Teacher Institute" 

CONTINUOUS ELECTRON BEAM ACCELERATOR FACILITY, Newport News, 
Virginia 

"Summer Institute for Teacher Enhancement" 



Environmental Protection Agency 

ENVIRONMENTAL PROTECTION AGENCY AND MIAMI UNIVERSITY, 
Cincinnati, Ohio 

"Hands-On Environmental Science" 
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Department of Commerce 

NATIONAL OCEANOGRAPHIC AND ATMOSPHERIC ADMINISTRATION 
LABORATORY, Boulder Colorado 

"NOAA Summer Science Institute" 

National Aeronautics and Spare Administration 

NASA JET PROPULSION LABORATORIES, Pasadena, California 
"Teacher Enhancement Program" 

NASA MARSHALL SPACE FLIGHT CENTER, Huntsville, Alabama 
"Summer Teacher Enrichment Program" 

Smithsonian 

SMITHSONIAN INSTITUTION, Washington, D.C. 

"Natural Science Institute for Teachers of Minority Students" 

Department of the Interior 

NATIONAL WETLANDS RESEARCH CENTER AND UNIVERSITY OF 
SOUTHWESTERN LOUISIANA, Lafayette, Louisiana 
"Microscopy of the Wetlands" 

Department of Agriculture 

CLEMSON UNIVERSITY AND PEE DEE RESEARCH AND EDUCATION 
CENTER, Florence, South Carolina 

"PEAK" Institute for Science in Agriculture" 
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FCCSET Summer Teacher Enhancement Institutes 
Application Form 

A. Complete the following: 

1 . Name of participating federal facility 

2. Collaborative partners 

3. Subject matter/theme of institute 

4. Grade level (s) of participants 

5. Number of participants 

6. National or local participants 

7. Commuting or residential institutes 

8. Funding request 

B. In the space below, describe your institute. 
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FCCSET Summer Institute Enhancement Institutes Application 

C. Briefly respond to the following questions: 

1. How does the institute take advantage of the resources unique to its federal 
facility? 

2. How has the need for this institute been demonstrated? 



3. What are the goals and objectives of the institute? 



4. How are the goals and objectives of the institute linked to reform efforts (i.e. , 
Statewide Systemic Initiative, National Council of Teachers of Mathematics 
Standards, etc.)? 



5. How does this institute model demonstrate the best of research and practice? 

6. What will participants be able to know as a result and practice? 

7. How will equity issues be addressed, that is, both the need for the inclusion of 
underrepresented teachers or teachers representing students traditionally 
underrepresented, and for a climate which is supportive of females? 

8. How will school systems support teachers participating in the institute? 
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FCCSET Summer Teacher Enhancement Institutes Application 

9. How will transfer of teacher learning from the institute to the classroom be 
ensured? 

10. How will the institute build leadership skills of participants? 

11. How will participants be selected? 

12. What academic year follow-up activities are planned? 

13. Will academic credit be available for participants? 

14. How will the institute be evaluated? 

D. Budget: 

1 . Develop a budget description, including all costs (stipends, staff, program 
development, etc.). Include all in-kind support. 
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Spring 1993 



Summer 1993 



Fall 1993 



Steps in Collecting Data Using the Template 



FCCSET/DOE Summer Institute Training (Spring) 

- Introduction to Template and its application to formative 
evaluation. 

- Technical assistance in developing skills in completing 
"intended" column, ie. (describing their programs intentions.) 



NCISE Conducts Site Visits 

- Check accuracy of "intended" column. 

- Complete "actual" column. 

- Completion of individual site visit reports which include 
Template, commendations, issues, and recommendations. 

FCCSET/DOE Conference 

- Re-visit Profiling. 
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A TOOL FOR PROFILING 
TEACHER DEVELOPMENT PROGRAMS 

The National Center for Improving Science Education 

with a Grant from 

The United States Department of Energy 
1993 



Purpose of Program Profiling 

Significant investments of time, energy, and resources are currently being made in a 
wide variety of programs whose aim is to improve education. The Department of 
Energy is one of the investors, with the mission of improving student learning in 
mathematics, science, and engineering technology through the enhancement of teacher 
knowledge and skills. A wide variety of programs are currently being funded by 
DOE that target precollege education, providing unique learning opportunities for both 
students and teachers. How good are these programs? Is the funding being invested 
wisely? What adjustments could be made to better use the available resources and, as 
they shrink, where should the cuts be made? 

In order to address these important impact questions, a prior question must be 
answered: what is actually happening in the programs? Without the answer to this 
question, it is impossible to know what it is that is causing either positive or negative 
learning outcomes. Likewise, it is impossible to know what to change for the 
purposes of improvement. The attached template was developed as a tool to help 
answer this question. 

The template has an additional purpose as well. Instead of allowing only for 
description of a program, it also allows for comparison of that program to best 
practice. The template identifies what research and experience report to be "what 
works" - elements that, when part of a program’s design, should enhance its 
effectiveness. Therefore, the template can also help answer another important 
question about a program: to what extent does it reflect best practice? Addressing 
this question helps program developers and managers identify components of their 
programs that could be the focus of improvement. 

The template is thus a formative evaluation tool, as well as one of several assessment 
instruments needed to evaluate the impact of a program. It is the key tool used in the 
first phase of evaluation of Department of Energy precollege education programs - 
called Program Profiling - and one of several data collection tools used in the second 
phase - Summative Evaluation. 
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Anatomy of a Template 

The template provides a format for "profiling" a program as it reflects best practice. 

It is formatted in three columns. The first column lists Components of Effective 
Practice, as determined by its developers from research and experience. The 
components are concisely worded, as appropriate to this kind of a tool, and some may 
find they need clarification and/or elaboration. For this purpose, a set of operational 
definitions and a reference list are included with the template. 

The second column provides the opportunity to describe the program as it is deigned 
to work, i.e., the Intended Program. The third column, then, is used to describe 
what actually happens when the program is functioning, the Actual Program. 

Information from these columns can be used to address the following questions: 

1. What is the best practice for these types of programs? 

2. To what extent is the program designed to reflect what is most effective? 

3. To what extent does the program in place reflect best practice? 

4. To what extent is the program’s design actually carried out? 

5. Where are the gaps? What can be improved? What is beyond our control? 

The last page of the template provides the opportunity to describe additional 
characteristics of the program and of its participants that may have an impact on its 
effectiveness. For example, certain program characteristics may work differently 
with different teachers (e.g., elementary vs. secondary), different school populations 
(e.g., urban vs rural), or different research foci of the sponsoring organization (e.g., 
theoretical vs hands-on). 

Development of Templates 

The templates for Department of Energy programs have been developed 
collaboratively, with the National Center for Improving Science Education playing a 
lead role, but with full participation from precollege education program managers and 
staff from DOE national Laboratories and other facilities. Early in the collaboration, 
it was determined that there was such a variety of programs that more than one 
template would be necessary to capture their important features. Several program 
"types" were identified in the first discussion - teacher research participation 
programs, student programs, systemic programs, and special programs - and it was 
decided that over the course of three years, a template would be developed for each. 

Template development begins with searching and reading literature on each program 
type, focusing particularly on what research and practice indicate to be components 
necessary for such programs to be effective. During meetings of DOE and Lab staff, 
readings (mainly syntheses) selected by the National Center were discussed and 
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components of effective practice identified through small group work. Center staff 
combined the work of groups into a draft template and circulated it for review. 

Once drafted, the template was piloted as the primary tool for profiling DOE 
programs across the Labs. The template for Teacher Development Programs was the 
second template developed. Five Labs where such programs were being conducted 
were selected for the pilot study, using a set of criteria developed collaboratively. 
Center staff read descriptive material about each Lab’s Teacher Development 
Program, and made two-day site visits during which program staff and teachers were 
interviewed and observed. A template was filled out as each Lab’s "profile"; at the 
same time, refinements were made in the template. 

Based on learnings from the pilot of how best to use the template to profile programs, 
the final draft of the template was then used by all Labs to profile their own Teacher 
Development Program. A summary of the profiles will be compiled by the National 
Center once that profiling is completed. 

This same process will be used to develop, pilot, and then put to use the template for 
each program type. 
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1993 SUMMER INSTITUTE SCHEDULES 



13 June - 2 July 


Clemson University and Pee Dee Research and Education 
Center, Florence, SC 


14 June - 9 July 


Superconducting Supercollider Laboratory 
Dallas, TX 


14 June - 9 July 


NASA Marshall Space Flight Center, Huntsville, AL 


14 June - 9 July 


NOAA Laboratory, Boulder, CO 


25 June - 23 July 


Oak Ridge Laboratory, Oak Ridge, TN 


27 June - 23 July 


Continuous Electron Beam Accelerator Facility, Newport 
News, VA 


28 June - 23 July 


Lawrence Berkeley Laboratory, Berkeley, CA 


6 July - 30 July 


Smithsonian Institution, Washington, DC 


6 July - 30 July 


Sandia National Laboratory, Livermore, CA 


6 July - 30 July 


Environmental Protection Agency and Miami University, 
Cincinnati, OH 


1 1 July - 1 Aug 


Pacific Northwest Laboratory, Richland, WA 


12 July - 6 Aug 


NASA Jet Propulsion Laboratories, Pasadena, CA 


12 July - 6 Aug 


NASA Marshall Space Flight Center, Huntsville, AL 


12 July - 6 Aug 


National Wetlands Research Center and University of 
Southwestern Louisiana, Lafayette, LA 


19 July - 13 Aug 


Sandia National Laboratory, Albuquerque, NM 


19 July - 13 Aug 


Argonne National Laboratory, Argonne, IL 
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1993 SUMMER INSTITUTE SITE VISITORS 

THE NATIONAL CENTER FOR IMPROVING SCIENCE EDUCATION 

STAFF AND ASSOCIATES 



Sally Crissman 
Joyce Kaser 
Paul Kuerbis 
Susan Loucks-Horsley 
Casey Murrow 
Senta Raizen 
Judy Sparrow 
Suzanne Stiegelbauer 
M. Jean Young 
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Pre-Participant Information Sheet 

Date: 

1. What grade level(s) do you currently teach? (Circle all that apply.) 

Middle School: 5 6 7 8 9 

High School: 9 10 11 12 

2. How many years have you taught the following? (Fill in the blank for all that 
apply.) 

Science 

Mathematics 

Vocational/ technology 

3. Which of the following best describes the ability of the students in your 
classes? (Indicate how many classes you teach in each category.) 



Fairly homogeneous and low in ability classes 

Fairly homogeneous and average in ability classes 

Fairly homogeneous and high in ability classes 

Heterogeneous, with a mixture of two or more ability levels classes 
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4. Rate the following in terms of how important you believe each is in order for 



your science/mathematics/technology teaching to be effective. 

(Circle one on each line) 







Should 
definitely 
not be 
included. . . 




Makes 

no 

difference 


Should 

definitely 

be 

included. . . 




a. 


integration of science/math/ techno logy 


1 


2 


3 


4 


5 


b. 


application of science/math/ 
technology in daily life 


1 


2 


3 


4 


5 


c. 


societal issues related to science/math/ 
technology 


1 


2 


3 


4 


5 


d. 


collaborative/cooperative learning 


1 


2 


3 


4 


5 


e. 


use of computers as an integral part of 
instruction 


1 


2 


3 


4 


5 


f. 


discussions of careers 


1 


2 


3 


4 


5 


g- 


writing about science/math/technology 


1 


2 


3 


4 


5 


h. 


use of hands-on/manipulative/ 
laboratory activities 


1 


2 


3 


4 


5 


i. 


use of scientific equipment/instruments 


1 


2 


3 


4 


5 


j- 


students conduct independent research 
projects 


1 


2 


3 


4 


5 


k. 


study of the nature of science and 
scientific inquiry 


1 


2 


3 


4 


5 


1. 


offer breadth of scope the science/ 
mathematics/technology subject by 
covering many topics 


1 


2 


3 


4 


5 


m. 


emphasis on science/mathematics/ 
technology facts 


1 


2 


3 


4 


5 


n. 


emphasis on mathematical reasoning 


1 


2 


3 


4 


5 


o. 


emphasis on fewer concepts (depth 
versus breadth 


1 


2 


3 


4 


5 


P- 


study of vocabulary before engaging 
in investigations on a topic 


1 


2 


3 


4 


5 


q- 


concrete experiences before abstract 
treatments 


1 


2 


3 


4 


5 


What else do you think is especially important to include that is not listed above? 
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5. Indicate the degree to which you agree or disagree with each of the following 
statements. 



(Circle one on each line) 







Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


I am comfortable with my current level 
of science/mathematics/technology 
knowledge 


I 


2 


3 


4 


b. 


I feel comfortable teaching science/ 
mathematics/technology 


1 


2 


3 


4 


c. 


I feel comfortable managing a class of 
students who are doing hands-on 
activities 


1 


2 


3 


4 


d. 


I feel comfortable demonstrating 
science/mathematics/technology 
principles to my students 


I 


2 


3 


4 


e. 


I feel confident in my ability to discuss 
science/mathematics/technology 
applications with my students 


1 


2 


3 


4 


f. 


I feel confident in my ability to inform 
my students of various 
science/mathematics/technology career 
opportunities 


1 


2 


3 


4 


g- 


I feel confident in my ability to help my 
students answer their own questions 


1 


2 


3 : 


4 


h. 


I feel confident in my ability to 
supervise my students’ research projects 


1 


2 


3 


4 




3 



ERIC 



72 



The National Center for Improving Science Education 



Appendix E 



6. Indicate your expectations in regard to the program (either during the 
program or as a result of your participation). 







1 


2 


3 






I do not 


I am not 


I expect 






expect this 


sure this 


this will 






will occur 


will occur 


occur 


a. 


observe scientific research in the laboratory 


1 


2 


3 


b. 


increase my science/mathematics/ 
technology content knowledge 


1 


2 


3 


c. 


increase my knowledge of applications in 
science/mathematics 


1 


2 


3 


d. 


gain new perspectives on how 
science/mathematics/technology should best 
be taught 


1 


2 


3 


e. 


learn about activities I can use in my 
classroom 


1 


2 


3 


f. 


develop activities I can use in my classroom 


1 


2 


3 


g- 


learn about how to use specific equipment 
and technologies in my classroom 


1 


2 


3 


h. 


learn laboratory skills that I can teach to my 
students 


1 


2 


3 


What other expectations do you have that are not listed above? 
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Program Evaluation Form 



Date: 

Institute Location: 

1. What grade level(s) do you currently teach? (Circle all that apply.) 

Middle School: 5 6 7 8 9 

High School: 9 10 11 12 

2. How many years have you taught the following? (Fill in the blank for all that 
apply.) 

Science 



Mathematics 



Vocational/technology 

3. Indicate the extent to which you agree with each of the following statements 
about what occurred during your participation in the program. 



(Circle one on each line) 





Not at 
All 






To a great 


a. 


I increased my science/ mathematics/ 
technology content knowledge 


1 


2 


3 


4 


b. 


I increased my knowledge of applications 
in science/mathematics 


1 


2 


3 


4 


c. 


I gained new perspectives on how science/ 
mathematics/technology should 
be best taught 


1 


2 


3 


4 


d. 


I learned about activities I can use in my 
classroom 


1 


2 


3 


4 


e. 


I learned laboratory skills that I 
can teach to my students 


1 


2 


3 


4 


What else occurred that you would like to mention? 
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4. Rate the following aspects of the program. 



(Circle one on each line) 





Poor 


Fair 


Good 


Very 

Good 


Excellent 


a. 


Program administration 


1 


2 


3 


4 


5 


b. 


Advance communication 


1 


2 


3 


4 


5 


c. 


Orientation 


1 


2 


3 


4 


5 


d. 


Availability of resources 


1 


2 


3 


4 


5 


e. 


Assistance provided by 
program staff 


1 


2 


3 


4 


5 


f. 


Workshop leaders 


1 


2 


3 


4 


5 


g- 


Interactions with other teachers 


1 


2 


3 


4 


5 


h. 


Interactions with scientists/ 
technicians 


1 


2 


3 


4 


5 


i. 


Receiving advice and support for 
sharing experience 


1 


2 


3 


4 


5 


j- 


Receiving support for extending 
experience to the classroom 


1 


2 


3 


4 


5 



List any specific strengths and weaknesses you would like the program staff to know 
about: 
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5. Rate each of the following in terms of its important for effective science/ 
mathematics/technology teaching to be effective. Answer for the subject 
(science, mathematics, technology) you spend the most time teaching. 

(Circle one on each line) 







Should 
definitely 
not be 
included... 




Makes 

no 

difference 


Should 

definitely 

be 

...included 




a. 


integration of science/math/technology 


1 


2 


3 


4 


5 


b. 


application of science/math/ 
technology in daily life 


1 


2 


3 


4 


5 


c. 


societal issues related to science/math/ 
technology 


1 


2 


3 


4 


5 


d. 


collaborative/cooperative learning 


1 


2 


3 


4 


5 


e. 


use of computers as an integral part of 
instruction 


1 


2 


3 


4 


5 


f. 


discussions of careers 


1 


2 


3 


4 


5 


g- 


writing about science/math/technology 


1 


2 


3 


4 


5 


h. 


use of hands-on/manipulative/ 
laboratory activities 


1 


2 


3 


4 


5 


i. 


use of scientific equipment/instruments 


1 


2 


3 


4 


5 


j- 


students conduct independent research 
projects 


1 


2 


3 


4 


5 


k. 


study of the nature of science and 
scientific inquiry 


1 


2 


3 


4 


5 


1 . 


offer breadth of scope the science/ 
mathematics/technology subject by 
covering many topics 


1 


2 


3 


4 


5 


m. 


emphasis on science/mathematics/ 
technology facts 


1 


2 


3 


4 


5 


n. 


emphasis on mathematical reasoning 


1 


2 


3 


4 


5 


o. 


emphasis on fewer concepts (depth 
versus breadth 


1 


2 


3 


4 


5 


P- 


study of vocabulary before engaging 
in investigations on a topic 


1 


2 


3 


4 


5 


q- 


concrete experiences before abstract 
treatments 


1 


2 


3 


4 


5 


What else do you think is especially important to include that is not listed above? 
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6. Indicate the degree to which you agree or disagree with each of the following 
statements. 



(Circle one on each line) 







Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


a. 


I am comfortable with my current level 
of science/mathematics/technology 
knowledge 


1 


2 


3 


4 


b. 


I feel comfortable teaching science/ 
mathematics/technology 


1 


2 


3 


4 


c. 


I feel comfortable managing a class of 
students who are doing hands-on 
activities 


1 


2 


3 


4 


d. 


I feel comfortable demonstrating 
science/mathematics/technology 
principles to my students 


1 


2 


3 


4 


e. 


I feel confident in my ability to discuss 
science/mathematics/technology 
applications with my students 


1 


2 


3 


4 


f. 


I feel confident in my ability to inform 
my students of various 
science/mathematics/technology career 
opportunities 


1 


2 


3 


4 


g- 


I feel confident in my ability to help my 
students answer their own questions 


1 


2 


3 


4 


h. 


I feel confident in my ability to 
supervise my students’ research projects 


1 


2 


3 


4 
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Appendix F 

Participant Survey Instruments: 
Tabulated data 3 



ERIC 



3 Total responses reflect input from ten out of fifteen Labs/facilities: two 
submit post surveys, one Lab did not submit any pre nor post survey results, 
misinterpreted questions and one Lab did not submit pre-survey results. 



Labs did not 
one Lab 
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FCCSET Institutes, Summer, 1993 
9/27/93 

Pre n = 328 Post n = 338 
Grade Level: pre 



Middle School 


148 


High School 151 


Both 20 Elementary 


4 


post 


166 


148 


16 


3 


Subject: pre 
Science 


279 


Math 158 


Technology 45 




post 


297 


170 


47 





Level of Satisfaction (Post) - Question #4 
(Figures in Percentages) 




i 


2 


3 


4 


5 








Poor 


Fair 


Good 


Very Good 


Excellent 


Mean 


SD 


a 


0.9 


3.3 


14.0 


35.8 


46.0 


4.2 


.87 


b 


3.3 


9.5 


18.8 


29.4 


39.0 


3.9 


1.12 


c 


0.9 


7.2 


18.6 


38.0 


35.3 


4.0 


.95 


d 


0 


3.0 


11.9 


33.9 


51.2 


4.3 


.80 


e 


0 


0.6 


4.4 


27.7 


67.3 


4.6 


.58 


f 


0.3 


0.9 


7.6 


34.7 


56.5 


4.5 


.70 


g 


0.3 


1.2 


8.9 


24.9 


64.7 


4.5 


.73 


h 


0.3 


1.8 


10.4 


30.1 


57.4 


4.4 


.77 


i 


0.6 


1.8 


9.3 


32.3 


56.0 


4.4 


.78 


j 


0.3 


1.8 


9.0 


29.7 


59.2 


4.5 


.76 
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Expectations (Pre) - Question #6 
(Figures in Percentages) 




1 

I do not 
expect this 
will occur 


2 

I am not 
sure this 
will occur 


3 

I expect 
this will 
occur 


Mean 


SD 


a 


4.8 


16.6 


78.6 


2.7 


.54 


b 


1.0 


7.2 


91.8 


2.9 


.32 


c 


1.0 


7.3 


91.7 


2.9 


.32 


d 


1.7 


10.8 


87.5 


2.8 


.40 


e 


0.7 


10.0 


89.3 


2.9 


.34 


f 


0.7 


13.4 


85.9 


2.8 


.38 


g 


2.1 


20.4 


77.5 


2.7 


.47 


h 


4.2 


20.1 


75.7 


2.7 


.54 



What Learned (Post) - Question #3 
(Figures in Percentages) 




i. 






. . .4 








Not at all 


To a great extent 


Mean 


SD 


a 


0 


8.8 


32.0 


59.2 


3.5 


.65 


b 


0 


3.8 


32.6 


63.6 


3.6 


.56 


c 


0.6 


4.7 


26.9 


67.8 


3.6 


.60 


d 


0 


4.2 


22.2 


73.6 


3.7 


.55 


e 


2.7 


13.3 


31.4 


52.6 


3.3 


.81 


1 = Not at all 












4 = To a great extent 
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Comfort Level (Pre) Question #5 Comfort Level (Post) - Question #6 

(Figures in Percentages) 



1 — Strongly Agree 2 — Agree 3 — Disagree 4 — Strongly Disagree 





1 


2 


3 


4 


Mean 


SD 




1 


2 


3 


4 


Mean 


SD 


a 


12.9 


45.8 


32.0 


9.3 


2.4 


.82 




19.2 


52.0 


23.1 


5.7 


2.1 


.79 


b 


35.0 


51.1 


10.2 


3.7 


1.8 


.76 




45.2 


47.0 


5.1 


2.7 


1.6 


.70 


c 


49.4 


42.3 


5.6 


2.7 


1.6 


.72 




62.7 


32.5 


2.4 


2.4 


1.4 


.66 


d 


40.3 


51.0 


5.6 


3.1 


1.7 


.71 




56.9 


36.4 


4.8 


1.9 


1.5 


.67 


e 


35.0 


48.6 


13.9 


2.5 


1.8 


.75 




44.3 


47.3 


6.0 


2.4 


1.7 


.70 


f 


23.4 


50.8 


21.8 


4.0 


2.1 


.78 




35.3 


51.8 


10.2 


2.7 


1.8 


.72 


g 


33.3 


55.5 


9.3 


1.9 


1.8 


.68 




45.9 


47.4 


4.5 


2.2 


1.6 


.67 


h 


30.4 


52.8 


.14.3 


2.5 


1.9 


.73 




46.1 


43.4 


8.0 


2.5 


1.7 


.73 
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1 2 3 4 5 

Should Makes no Should 

definitely Difference definitely 

not be be 

included... included... 


Best Practice (Pre) - Question #4 




Best Practice (Post) - Question #5 




Mean 


SD 


Mean 


SD 


a 


4.7 


.49 


4.7 


.50 


b 


4.8 


.45 


4.8 


.46 


c 


4.5 


.60 


4.6 


.66 


d 


4.4 


.64 


4.6 


.61 


e 


4.4 


.67 


4.4 


.65 


f 


4.3 


.68 


4.3 


.73 


g 


4.3 


.65 


4.4 


.71 


h 


4.8 


.39 


4.9 


.41 


i 


4.7 


.52 


4.7 


.49 


j 


4.4 


.69 


4.4 


.67 


k 


4.4 


.65 


4.5 


.66 


1 


4.0 


.93 


4.1 


.96 


m 


3.9 


.90 


3.9 


.96 


n 


4.4 


.66 


4.2 


.74 


o 


4.0 


.97 


4.0 


1.00 


P 


3.7 


1.00 


3.8 


.98 


q 


4.2 


.78. 


4.3 


.86 


1 = Should definitely not be included 
5 = Should definitely be included 
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Follow-up Site Visits Schedule 



12 April 1994 


Smithsonian, Washington, DC 


22 April 1994 


Continuous Electron Beam Accelerator Facility, 
Newport News, VA 


23 April 1994 


NOAA Laboratory, Boulder, CO 


9 May 1994 


Clemson University and Pee Dee Research and Education 
Center, Florence, SC 


13 May 1994 


NASA Marshall Space Flight Center, Huntsville, AL 


18 May 1994 


Sandia National Laboratory, Albuquerque, NM 


18 May 1994 


Argonne National Laboratory, Argonne, IL 


15 June 1994 


Lawrence Berkeley Laboratory, Berkeley, CA 



1993 Summer Institute Follow-up Site Visitors 

The National Center for Improving Science Education 
Staff and Associates 



Sally Crissman 
Simon Hawkins 
Joyce Kaser 
Paul Keurbis 
Pat Price 
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Guidelines for Follow-up Site Visitors 
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Guidelines for FCCSET/NSF Follow-up Site Visitors 

The assessment study the Center conducted between June and December 1993 
documented the FCCSET/DOE Summer Institute programs’ intentions to conduct 
follow-up activity, but could not document what activities were actually implemented, 
nor their quality and results because most of the activities had not yet occurred. We 
subsequently were commissioned to examine the follow-up activities that were actually 
implemented and observe others in process to ascertain their quality and usefulness. 
Results should enable the program operators to improve the programs and increase the 
likelihood that the programs have a positive effect on teaching and learning. 

There are two purposes guiding this work: 

1. To conduct a study of the follow-up implemented by the 15 Summer Institutes 
and report what teachers have done as a result of their experiences. 

2. To identify what follow-up activities were actually done after the Institute to 
support the learning of Institute participants — i.e., document what follow-up 
activities were done, what they entailed (e.g., "looked like"), and what impact 
they had. 

Information To Be Gathered at Site Visit — Due One Week after Site Visit 

1. Complete template Section 5 on Follow-up: Ask the site director to review the 
intended column for accuracy; then complete the actual column. 

2. Complete template questions l.f. regarding how the program administration 
ensured effective program follow-up. Ask how the follow-up is organized. 

3. Have director fill out the Descriptive and Context Information page at the back 
of the template. This was not completed in the first study, so we are trying to 
get that information now. 

4. Collect as much information as possible as to the nature of follow-up 
activities; the impact or perceptions of the teachers served; any anecdotes, etc. 
that will help us paint a picture of what actually has taken place. 

5. Prepare a report to include completed template (section 5 and l.f), 
commendations, recommendations, and issues of the follow-up section, 
capturing any "other" information pertinent. Report should be "meaty," at 
least two pages in length. 

6. Submit this report, completed template, and any materials the Institute has 
given you as samples of their follow-up work to Sandra Thibodeau at the 
Andover office within one week of your site visit. 
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Evaluation Report: FCCSET/DOE 1993 Summer Institutes 



Sample Follow-Up Questions 

Used in telephone and personal interviews of participants in 
1993 FCCSET Summer Institutes 

What activities were you aware of that followed up the Summer Institute? 

Which ones did you participate in? 

Could you describe them briefly? 

What did you find most valuable about them? 

What would you have changed, added, or eliminated? 

How have you been able to use the experiences of the Summer Institute in your own 
teaching? 

Has your school supported any changes (new ideas) you brought back from the 
Summer Institute? 

Were you able to share ideas with colleagues at your school or elsewhere? 

Have you received direct follow-up help from the Institute staff? 

What has that consisted of? 

What more should the Institute staff do? 

Are there aspects of the Institute and its follow-up activities that are particularly worth 
mentioning that we have not discussed already? If so, what are they? 



103 



o 

ERIC 



92 



The National Center for Improving Science Education 



Appendix I 




Questions That May Be Useful for the Follow-up Site Visit 
General Follow-up 

1 . What were the intended formal follow-up activities? 

2. What were the actual formal follow-up activities? 

3. What percentage of the teachers were involved? 

4. Who initiated the activities (teachers, scientists, program staff, etc.)? 

5. Who designed the activities? 

6. Who implemented the activities? 

7. What was the content of the activities (expanding work from the summer, teaching 
new skills/resources, sharing implementation strategies, etc.)? 

8. What were some unplanned follow-up activities:? (See above questions.) 

9. How did they measure the follow-up’s effectiveness? 

10. What problems did they run into with follow-up? 

1 1 . What (if any) strategies did they pursue (or contemplate pursuing in the future) to 
overcome these problems? 

Observed Follow-up Activity 

1. What is (are) the group size(s)? 

2. What are the goals? 

3. How are they measured? 

4. What is the mode of discourse (lecture, general discussion, small groups, hands-on 
activity, etc.)? 

5. What is the teachers’ role? 

6. What did they do (what actually happened)? 
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